Image processing apparatus, image processing method, program executing image processing method, and storage medium

ABSTRACT

An image processing apparatus including an area sensor unit reading image data items corresponding to frames from an original image, a correction unit correcting the inclinations of the image data items, a high-resolution conversion unit acquiring image data with a resolution higher than the pixel sensor&#39;s resolution through interpolation, a maximum frame number storage unit storing data of the maximum number of frames of the acquired image data items, a resolution setting unit setting a resolution for outputting the original image, a necessary frame number acquisition unit acquiring the number of frames necessary to perform the high resolution conversion based on the setting result, a read frequency calculation unit calculating the frequency of reading the original image through the necessary frame number acquisition unit and the maximum frame number storage unit, and a read frequency control unit reading the determined frequency and the original image is provided.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing apparatus, a methodof controlling the image processing apparatus, and a program executingthe image processing method, a storage medium program, and a storagemedium.

2. Description of the Related Art

The technology referred to as “super-resolution processing and superresolution conversion”, which allows for increasing a resolution byusing image data with a predetermined resolution by as much as aplurality of frames, has been available. The use of the above-describedtechnology allows for converting a low-resolution image into ahigh-resolution image and obtaining a high-resolution image through aknown device (“Super Resolution Processing by Plural Number of LowerResolution Images” (Ricoh Technical Report No. 24, November 1998)).

For performing the super-resolution technology, data items of anoriginal image should be prepared, the image data items corresponding toa plurality of frames. The original-image reading positions of theindividual image data items are a little different from one another on ascale measured in sub pixels (the sub pixel is smaller than a singlepixel). Therefore, the super-resolution technology is widely used in thearea of video processing or the like.

However, for performing the super-resolution processing, image datashould be prepared by as much as a plurality of frames, so as togenerate the image corresponding to a single pixel of a high-resolutionimage. Consequently, the data amount and the calculation amount areincreased.

In the past, therefore, the number of frames for which thesuper-resolution processing is performed is determined based on the sizeof an image area of interest so that the calculation amount isdecreased, as disclosed in Japanese Patent Laid-Open No. 2006-092450.

However, the number of images for which the super-resolution processingis performed is determined only for the area of interest according tothe above-described known technology. Therefore, the number offrame-image data items that should be prepared for performing thesuper-resolution processing should be obtained on the entire image areain advance.

Further, when the super-resolution processing is used for amultifunction peripheral (MFP) which is an image processing apparatus,line sensors are usually used as a reader provided in the MFP, ascanner, and so forth.

That is to say, a single read frame is obtained through a single readoperation. According to the above-described reader, an original image isread through groups of pixel sensors that are horizontally arranged inthe main scanning direction with distances therebetween, the value ofeach of the distances corresponds to an integer multiple of the pixel.It is difficult for the above-described reader to read the originalimage while providing shifts that are as small as a sub pixel betweenthe positions where the original image is read in the main scanningdirection.

Therefore, an area sensor is installed in the image processingapparatus, so as to be inclined from a reference installation position,frame image data can be acquired through a single read operation, whilethe positions of pixels for reading are little shifted from one anotherin the main scanning direction and/or the sub scanning direction. Inthat case, however, low-resolution frame image data corresponding to aplurality of read frames is used irrespective of conditions foroutputting image data. Therefore, it is often difficult to reproduce anoutput image with a demanded quality based on the read low-resolutionframe image data.

SUMMARY OF THE INVENTION

Accordingly, the present invention provides an image processingapparatus and an image processing method used for the image processingapparatus, so as to respond to a demand for the output image quality andachieve the super-resolution processing.

Therefore, an image processing apparatus according to an aspect of thepresent invention includes an area sensor unit configured to read imagedata items corresponding to a plurality of frames from an originalimage, the image data items having at least one shift corresponding toless than a single pixel, a correction unit configured to correct theinclination of each of the image data items acquired through thearea-sensor unit, a high-resolution conversion unit configured toacquire image data with a resolution higher than that of a pixel sensorby performing interpolation processing by using the corrected image dataitems, a maximum frame number storage unit configured to store data ofthe maximum number of at least one frame of the image data itemsacquired through the area sensor unit, a resolution setting unitconfigured to set a resolution used to output the original image, anecessary frame number acquisition unit configured to acquire a numberof at least one frame necessary to perform the high resolutionconversion based on a result of the setting made by the resolutiondetermining unit, a read frequency calculation unit configured tocalculate a frequency of reading the original image through thenecessary frame number acquisition unit and the maximum frame numberstorage unit, and a read frequency control unit configured to read thefrequency determined by the read frequency calculation unit and theoriginal image.

The present invention allows for acquiring low-resolution frame imagedata necessary to perform the super-resolution processing in response toa demand for an output image quality and outputting a high-resolutionimage reproduced with difficulty through a single read operation.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an outline drawing of an image processing apparatus.

FIG. 2 shows the configuration of a read unit of the image processingapparatus.

FIG. 3 is a block diagram describing the configuration of a controllerprovided in the image-processing apparatus.

FIG. 4 is a block diagram showing the internal configuration of ascanner-image-processing unit.

FIG. 5 shows exemplary images obtained through a scanner unit.

FIG. 6 is a block diagram showing the internal configuration of aprinter image processing unit.

FIG. 7 shows the configuration of an area sensor.

FIG. 8 shows an original image read by the area sensor.

FIG. 9 shows a method of acquiring line image data.

FIG. 10 shows another method of acquiring line image data.

FIG. 11 shows another method of acquiring line image data.

FIG. 12 shows another method of acquiring line image data.

FIG. 13 shows image data read by a line sensor provided in the areasensor.

FIG. 14A is a configurational diagram showing the area sensor mounted ina slanting direction.

FIG. 14B is a configurational diagram showing the area sensor mounted inanother slanting direction.

FIG. 15 shows a method of acquiring line image data through the inclinedarea sensor.

FIG. 16 shows another method of acquiring line image data through theinclined area sensor.

FIG. 17 shows another method of acquiring line image data through theinclined area sensor.

FIG. 18 shows image data read by a line sensor provided in the inclinedarea sensor.

FIG. 19 is a flowchart provided to schematically describe operationsperformed to execute super-resolution processing mode setting processingrelated to a first embodiment of the present invention.

FIG. 20 shows a graphical user interface (GUI) showing an exemplaryoperation unit provided to determine an output resolution shown in FIG.19 according to the first embodiment.

FIG. 21 shows a table providing information about the correspondencebetween the output resolution shown in FIG. 19 and the number oflow-resolution images according to the first embodiment.

FIG. 22 provides a detailed explanation of super-resolution processing.

FIG. 23 provides another detailed explanation of the super-resolutionprocessing.

FIG. 24 shows a method of dividing an area sensor according to the firstembodiment.

FIG. 25 is a diagram showing the configuration of an ADF provided in thescanner unit shown in FIG. 1 according to a second embodiment of thepresent invention.

FIG. 26 is a diagram showing the configuration of a scanner main bodyprovided in the scanner unit shown in FIG. 1 according to the secondembodiment.

FIG. 27 is a flowchart schematically illustrating operations performedto execute original read processing shown in FIG. 19 according to thesecond embodiment.

DESCRIPTION OF THE EMBODIMENTS

A first embodiment of the present invention will be described. In thefirst embodiment, a method of generating a high-resolution image byusing an area sensor will be described, where the method is provided foran image processing apparatus including a color scanner.

Appearance of Image Processing Apparatus

FIG. 1 shows the appearance of an image processing apparatus 1. Theimage-processing apparatus 1 is divided into a scanner unit 11 readingan original image, a printer unit reproducing read image data, and anoperation unit 13 specifying various operation settings of theimage-processing apparatus 1. The scanner unit 11 converts informationabout the original image into an electric signal by transmittingreflected light obtained by exposing and scanning an image shown on theoriginal image to a charge coupled device (CCD). Further, the scannerunit 11 converts the electric signal into brightness signals of red (R),green (G), and blue (B), and transmits the brightness signals to acontroller 20 which will be described later with reference to FIG. 3, asimage data.

Here, the original image is placed in a tray 14 of an original feeder15. When a user instructs to start reading data from the operation unit13, the controller 20 transmits an original-image-read instruction tothe scanner 11. Upon receiving the original-image-read instruction, thescanner unit 11 feeds original images one at a time from the tray 14 ofthe original feeder 15 and performs a read operation for the originalimage. Here, the method of reading the original image may be a system ofplacing the original image on the surface of a glass plate (not shown)and moving an exposure unit so that the original image is scannedinstead of an automatic-feeding system achieved by the original feeder15.

The printer unit 12 is an image processing apparatus forming the imagedata transmitted from the controller 20 on a sheet. Here, even though animage-processing system used in the first embodiment is anelectrophotography system using a photoconductive drum and/or aphotoconductive belt, the present invention can be achieved by usinganother system without being limited to the above-describedelectrophotography system. For example, an inkjet system which allowsfor discharging ink from a micro-nozzle array and printing data on asheet may be used. Here, the printer unit 12 is provided with aplurality of sheet cassettes 17, 18, and 19 which allows for makingselections from different sheet sizes and/or different sheetorientations. After printing is finished, the sheet is discharged into adischarge tray 16.

(Configuration of Read Unit of Image Processing Apparatus)

FIG. 2 shows an exemplary image read unit of an MFP using theabove-described embodiment. FIG. 2 shows a reader main body 201 and anautomatic-original feeder 202 holding an original image 203 andtransferring the original image to an original-read position whenmoving-original reading is performed. FIG. 2 also shows a glass plate,that is, an original plate 204 on which the original image 203 is placedwhen original-plate reading is performed.

FIG. 2 also shows a read unit 205 including a read device reading theoriginal image 203, a device picking up the original image 203, a lightsource 206 including a white-light source such as a xenon tube, andmirrors 207, 208, 209, 210, and 211. When the original image 203 isirradiated with light transmitted from the light source 206, the lightreflected from the original image is transferred to an image-pickupelement 213 by the mirrors 207 to 211. FIG. 2 also shows a lens 212condensing the light that is reflected from the original image and thatis reflected from the mirror 211 to the width of the image-pickupelement 213.

Detailed Description of Controller

FIG. 3 is a block diagram describing the configuration of a controller20 provided in the image-processing apparatus 1 in more detail.

The controller 20 is electrically connected to the scanner unit 11 andthe printer unit 12. On the other hand, the controller 20 is connectedto an external device or the like via a local-area network (LAN) 21and/or a wide-area network (WAN). Consequently, image data and/or deviceinformation can be input and/or output.

A CPU 301 has control over access to various devices to which thecontroller 20 is currently connected based on a control program or thelike stored in a read-only memory (ROM) 303. Further, the CPU 301 hascontrol over various processing procedures performed in the controller20.

A random-access memory (RAM) 302 is a system-work memory provided sothat the CPU 301 can operate and is a memory provided to temporarilystore image data. The above-described RAM 302 includes a static RAM(SRAM) that retains stored data after the power is turned off and adynamic RAM (DRAM) from which stored data is deleted after the power isturned off.

The ROM 303 stores a boot program or the like of the image-processingapparatus 1. A hard disk drive (HDD) 304 can store system softwareand/or image data.

An operation-unit interface (I/F) 305 is provided to connect a systembus 310 to an operation unit 13. The above-described operation-unit I/F305 receives image data which is to be displayed on the operation unit13, the image data being transmitted from the system bus 310, andtransmits the image data to the operation unit 13, and transmitsinformation transmitted from the operation unit 13 to the system bus310.

A network I/F 306 is connected to the LAN 21 and the system bus 310, soas to input and output information. A modem 307 is connected to the WAN22 and the system bus 310, so as to input and output information. Abinary-image-rotation unit 308 changes the orientation of image databefore the image data is transmitted. Abinary-image-compression-and-expansion unit 309 changes the resolutionof the image data to a predetermined resolution and/or a resolutionfitted to the capacity of the transmission destination before the imagedata is transmitted. When the image data is compressed and/or expanded,the Joint Bi-level Image Experts Group (JBIG) system, the ModifiedModified Read (MMR) system, the Modified Read (MR) system, the ModifiedHuffman (MH) system, and so forth are used. An image bus 330 is atransmission path provided to transmit and/or receive image data andincludes a Peripheral Components Interconnect (PCI) bus and/or anInstitute of Electrical and Electronic Engineers (IEEE) 1394.

A scanner-image-processing unit 312 corrects, processes, and edits imagedata transmitted from the scanner unit 11 via a scanner I/F 311. Here,the scanner-image-processing unit 312 determines to which of a colordocument, a monochrome document, a text document, a photo document, andso forth the transmitted image data corresponds. Then, thescanner-image-processing unit 312 attaches information about a result ofthe above-described determination to the image data. The above-describedattached information is referred to as attribute data. Details ofprocessing procedures performed in the scanner-image-processing unit 312will be described below.

A compression unit 313 receives and divides the image data into 32 pixelby 32 pixel blocks. Here, the 32 pixel by 32 pixel-image data isreferred to as tile data. The area corresponding to the above-describedtile data, the area being shown on an original (a paper medium beforebeing read), is referred to as a tile image. Further, information aboutthe average brightness of the 32 pixel by 32 pixel block and thepositions of the coordinates of the tile image on the original is addedto the tile data as header information. Further, the compression unit313 compresses image data including a plurality of the tile-data items.An expansion unit 316 expands the image data including the plurality oftile-data items, and performs raster development of the image data.Then, the expansion unit 316 transmits the image data to aprinter-image-processing unit 315.

The printer-image-processing unit 315 receives the image datatransmitted from the expansion unit 316, and performs image processingfor the image data while referring to the attribute data attached to theimage data. The image data that had been subjected to the imageprocessing is transmitted to the printer unit 12 via a printer I/F 314.Details of processing procedures performed in theprinter-image-processing unit 315 will be described below.

An image-change unit 317 performs predetermined change processing forthe image data. The image-change unit 317 includes units shown below.

An expansion unit 318 expands image data transmitted thereto. Acompression unit 319 compresses image data transmitted thereto. Arotation unit 320 rotates image data transmitted thereto. Avariable-magnification unit 321 performs resolution-change processing(e.g., from 600 dpi to 200 dpi) for image data transmitted thereto. Acolor-space-change unit 322 changes the color space of image datatransmitted thereto. The color-space-change unit 322 can perform knownbackground-eliminating processing, known logarithmic (LOG)transformation processing (RGB→CMY), known output-color-correctionprocessing (CMY→CMYK), and so forth by using a matrix and/or a table. Abinary-to-multilevel-conversion unit 323 converts two-level image datatransmitted thereto into 256-level image data. Amultilevel-to-binary-conversion unit 324 converts 256-level image datatransmitted thereto into two-level image data by using a systemincluding error-diffusion processing or the like.

A merging unit 327 generates a single image-data item by merging twoimage-data items transmitted thereto with each other. When the twoimage-data items are merged with each other, one or more of thefollowing methods may be used, for example, to determine themerge-brightness value of a pixel. The merge-brightness value of thepixel refers to the brightness value of the pixel after the merging. Themethods include the method of determining the average brightness valueof the pixels which are to be merged with one another to be themerge-brightness value of the pixel, the method of determining thehighest brightness value among the pixels which are to be merged withone another to be the merge-brightness value of the pixel, and themethod of determining the lowest brightness value among the pixels whichare to be merged with one another to be the merge-brightness value ofthe pixel. Further, a method of determining the brightness values usedafter the merging by performing OR operation, AND operation, exclusiveOR operation, and so forth for the pixels which are to be merged withone another may also be used. Each of the above-described mergingmethods is widely known. A thinning unit 326 performs resolutionconversion by thinning out the pixels of image data transmitted thereto,and generates the image data corresponding to one second, one fourth,one eighth, and so forth of the image data transmitted to the thinningunit 326. A moving unit 325 adds and/or deletes a margin to and/or fromimage data transmitted thereto.

A raster image processor (RIP) 328 receives intermediate data generatedbased on Page Description Language (PDL) code data transmitted from aprint server (not shown) or the like, generates bitmap data(multilevel), and compresses the bitmap data through a compression unit329.

Detailed Description of Scanner-Image-Processing Unit 312

FIG. 4 shows the internal configuration of the scanner-image-processingunit 312. The scanner-image-processing unit 312 receives image dataincluding RGB brightness signals, where each of the signals is an 8-bitsignal, transmitted from the scanner unit 11 via the scanner I/F 311.The brightness signal is converted into a standard brightness signalindependent of the filter color of the CCD through a masking-processingunit 401.

A filter-processing unit 403 arbitrarily corrects the space frequency ofimage data transmitted thereto. The filter-processing unit 403 performscalculation processing for the image data transmitted thereto by using a7-by 7-pixel matrix, for example. Incidentally, in the case where acopying machine is used, the user of the copying machine can select textmode, photo mode, or text/photo mode, as copy mode by operating theoperation unit 13. If the text mode is selected by the user under theabove-described circumstances, the entire image data is text-filteredthrough the filter-processing unit 403. Further, if the photo mode isselected, the entire image data is photo-filtered. Still further, if thetext/photo mode is selected, the filter-processing unit 403 switchesbetween the filters adaptively for each pixel in accordance with atext/photo determining signal (part of the attribute data) that will bedescribed below. Namely, which of the photo filter and the text filtershould be used is determined for each pixel. Further, a coefficientwhich allows for smoothing high-frequency components only is set to thephoto filter so that the roughness of an image is reduced. Stillfurther, a coefficient which allows for performing strong edgeemphasizing is set to the text filter so that the sharpness of acharacter is increased.

A histogram-generation unit 404 samples the brightness data of each ofpixels included in image data transmitted thereto. More specifically,brightness data that falls within a rectangular area surrounded by astart point and an end point that are individually specified in themain-scanning direction and the sub-scanning direction is sampled atregular pitches in the main-scanning direction and the sub-scanningdirection. Then, the histogram-generation unit 404 generates histogramdata based on the sampling result. The generated histogram data is usedto estimate the ground level when performing the background-eliminatingprocessing. An input-side-gamma-correction unit 405 converts thehistogram data into brightness data having a nonlinear property by usinga table or the like.

A color/monochrome-determining unit 406 determines whether each of thepixels included in image data transmitted thereto is chromatic orachromatic, and attaches data of the determination result to the imagedata, as a color/monochrome-determining signal (part of the attributedata).

A text/photograph-determining unit 407 determines whether each of thepixels included in the image data constitutes a character, a grid ofdots, a character shown in the grid of dots, or a solid image based onthe values of the individual pixels and the values of pixels surroundingthe individual pixels. If the pixel does not constitute any of thecharacter, the grid of dots, the character, and the solid image, thepixel constitutes a white area. Data of the determination result isattached to the image data, as a text/photo-determining signal (part ofthe attribute data).

A super-resolution-processing unit 402 executes super-resolutionprocessing for image data transmitted thereto. Further, in the casewhere a copying machine is used, the user of the copying machine canselect super-resolution-processing mode by operating the operation unit13.

Although details on the super-resolution-processing mode will bedescribed later, the super-resolution processing is performed undercertain conditions.

First, image data of an original image should be prepared by as much asa plurality of frames, where the positions where the original image isread are a little shifted in the main-scanning direction and/or thesub-scanning direction with reference to image data of the originalimage read with the sensor resolution of a reader.

Namely, the image data items corresponding to consecutive frames shouldbe prepared, where the positions of a document read by a sensor are alittle shifted in the main-scanning direction and/or the sub-scanningdirection from the document position of reference image data.

Further, when the image data items corresponding to the frames are read,the amount of the shift of the read position of the original image, theshift existing between image-data items obtained by adjacent sensors,should be less than a single pixel (sub pixel) in the main-scanningdirection and/or the sub-scanning direction. The shift of the readposition may be a shift of which amount is less than a single pixel, theshift being obtained by subjecting an integral multiple of the positionshift to offset correction. Hereinafter, data which is read at the timewhen an original image including a single screen image (frame) isscanned and which constitutes the original image corresponding to theabove-described single screen image (frame) is referred to as“frame-image data”. Further, the position of a pixel read on theoriginal image is referred to as a “phase”.

Further, a phenomenon in which the phase is shifted is referred to as“phase is shifted”, and the shift of the position of a pixel for readingis referred to as a “phase shift”.

Further, the value of a low resolution used in the above-describedembodiment is not limited to 300 dpi. Namely, the low resolutionindicates the resolution of an image output from the image processingapparatus 1 performing ordinary printing.

The main-scanning direction is a direction orthogonal to a direction inwhich the read unit 205 is moved toward the original image placed on anoriginal plate when the document is read through a scanner. Then, thelandscape orientation of the read original image is referred to as the“main-scanning direction”, as indicated by an arrow A shown in FIG. 8.

Similarly, the sub-scanning direction is a direction parallel to thedirection in which the read unit 205 is moved. Then, the portraitorientation of the read original image is referred to as the“sub-scanning direction”, as indicated by an arrow B shown in FIG. 8.

According to the above-described embodiment, an area sensor is placed ina slanting direction, which allows for acquiring a plurality of imagesof which phases are shifted with reference to the main-scanningdirection and the sub-scanning direction for each RGB channel.

FIG. 5 shows exemplary images obtained through the above-describedembodiment. The phase of each of acquired images 501, 502, 503, 504, and505 is shifted with reference to the main-scanning direction and thesub-scanning direction.

(Area Sensor)

In the above-described embodiment, the sensor reading image data isprovided as an area sensor. The area sensor is an image-pickup elementused for a digital camera or the like. In contrast to theabove-described sensor provided for each line, pixel sensors configuredto read data are two-dimensionally arranged.

FIG. 7 shows the configuration of the above-described area sensor, thatis, an area sensor 701.

The area sensor 701 includes a pixel sensor 702. The pixel sensor 702includes pixel sensors provided in H pixels arranged in the long-sidedirection and those provided in L pixel sensors arranged in theshort-side direction. Each of the pixels may include a pixel sensorready for an RGB color image, the pixel sensor being achieved bydividing the pixel sensor of the above-described pixel into four parts.Further, the H pixels may be equivalent to the L pixels (the longside=the short side). The resolution of the area sensor is determinedbased on the distance N between the pixel sensors.

An area sensor used for a high-resolution digital camera includes alarge number of pixels so that the number of pixel sensors arranged inthe long-side direction and those arranged in the short-side directionis increased. For example, some of digital cameras including about tenmillion pixels include 3800 pixels, as pixel sensors arranged in thelong-side direction, and 2800 pixels, as pixel sensors arranged in theshort-side direction.

Generally, in the case where an area sensor is used for a camera or thelike, the area sensor captures and picks up transmitted image data, asdata of a two-dimensional area.

Namely, the area sensor picks up image data by using two-dimensionallyarranged pixel sensors for a single imaging. When the area sensor ismounted on the reader, the pixel sensors are arranged straight so thatpicked-up original image data is not distorted in a horizontal directionand a vertical direction.

Consequently, the pixel sensors are arranged so that when the picked-uporiginal image is reproduced, the reproduced image is not shifted in aslanting direction at all.

For example, when the area sensor is installed in an ordinary camera,image data read by pixel sensors arranged on a line indicated by a blackframe 703 becomes image data constituting the top end part of apicked-up object image.

At that time, the read image data is not inclined in the direction inwhich the line is formed.

Similarly, image data read by pixel sensors arranged on a line indicatedby a black frame 704 is image data located at a position different fromthe position of the picked-up object image read in the black frame 703.Namely, the position of the picked-up object image read in the blackframe 704 is below the position of the picked-up object image read inthe black frame 703 in a vertical direction. Accordingly, image dataread by pixel sensors arranged on a line indicated by a black frame 705is located at a position four pixels below the position of the picked-upobject image in a vertical direction, the picked-up object image beingread by the pixel sensors provided in the black frame 703.

When using the area sensor of the digital camera in the above-describedmanner, the image data is picked up as the two-dimensional area.Therefore, all of the pixel sensors constituting the area sensor pick upthe picked-up object image from different positions. However, the methodof using the area sensor provided in the apparatus used in theabove-described embodiment is different from the method of using thearea sensor provided in the above-described digital camera.

First, the area sensor shown in FIG. 7 is installed at a referenceinstallation position defined on the reader.

When an original image is placed at a position specified on the originalplate 204 shown in FIG. 1, light that is applied from a light source tothe original image and that is reflected from the original image iscondensed by the sensor, where the light source runs in parallel withthe same orientation as the portrait orientation of the original imageunder the original image. The reflected light is captured so that thereflected light is not inclined toward the sensor. The reflected lightacquired as the image data corresponding to a single line by subjectingthe light source to the parallel scanning is condensed in parallel withthe landscape orientation (the long-side direction) of the sensor shownin FIG. 7.

Therefore, the sensor is installed at a position determined so that thesensor can capture the original image with approximately no inclination.

The above-described installation position at which the sensor isinstalled, so as to achieve an output of the original image, is referredto as the “reference installation position” of the sensor.

In the following description, the sensor includes the pixel sensors of20 pixels arranged in the long-side direction and those of 10 pixelsarranged in the short-side direction for the sake of simplicity ofdescription. Of course, it may be configured that the number of thepixel sensors arranged in the long-side direction is equivalent to thatof the pixel sensors arranged in the short-side direction. Further, theabove-described pixel-sensor numbers are determined to describe the useand the configuration of the area sensor of the above-describedembodiment, and are not limited to the numbers of the pixel sensorsshown in FIG. 7.

In practice, the numbers of the pixel sensors may be the same as thoseof the pixel sensors used in the digital camera.

The read unit 205 including the image-pickup element 213, that is, thearea sensor mounted on the reader is driven in the direction of an arrowshown in FIG. 2 so that the original image 103 placed on the originalplate 204 is read.

Namely, the read line sensors 704 and 705, where each of the sensors 704and 705 is a group of the pixel sensors, are handled, as is the casewith the above-described line sensor. Consequently, a read operation isperformed.

Next, how the image data read by the read line sensors 704 and 705 arehandled will be described. FIG. 8 shows an original image read in thefollowing description.

A grid shown in FIG. 8 corresponds to the resolution of pixel sensorsincluded in the read line sensor 704 and/or the read line sensor 705.

At the same time as when the read unit 205 is driven and moved in thesub-scanning direction under the original plate 204, image-data itemstransmitted to the read line sensors 704 and 705 are read in sequence.

That is to say, part of the document data, the part corresponding to theline width representing the position of the read unit 205, is read everymoment.

The process of reading the original image will be described. When theread unit 205 is moved in the sub-scanning direction under the originalplate 204, the diagonally shaded areas of the original image shown inpart (a) of FIG. 9, part (a) of FIG. 10, part (a) of FIG. 11, and part(a) of FIG. 12 are irradiated with light emitted from the light source.

First, the diagonally shaded area shown in part (a) of FIG. 9 isirradiated with light emitted from the light source at a certain moment.Then, the area sensor detects the light and the original imagecorresponding to the line-width part irradiated with the light.

At that time, for example, the line sensor 704 detects an exemplaryimage-data item shown in part (b) of FIG. 9. At the same time, the linesensor 705 detects an exemplary image-data item shown in part (c) ofFIG. 9.

The read positions of the two image-data items are shifted from eachother because the two line sensors are installed with a physicaldistance therebetween in the sub-scanning direction.

Then, original images for reading are handled as image-data itemsvarying from one read-line sensor to another and the image-data itemsare separately stored in storage mediums such as memories shown in parts(d) and (e) of FIG. 9.

Next, at the same time as when the sensor unit 205 and the light sourceare moved, the position of the original image detected by the linesensor is changed, as shown in part (a) of FIG. 10. Then, the linesensor 704 detects an image shown in part (b) of FIG. 10 and the linesensor 705 detects an image shown in part (c) of FIG. 10.

Then, original images for reading are handled as image-data itemsvarying from one read-line sensor to another and the image-data itemsare separately stored in storage mediums such as memories shown in parts(d) and (e) of FIG. 10.

Similarly, when a part of the original image is read, the partcorresponding to a position shown in part (a) of FIG. 11, image-dataitems shown in parts (b) and (c) of FIG. 11 are stored in storagemediums such as memories shown in parts (d) and (e) of FIG. 11.

Further, when a part of the original image is read, the partcorresponding to a position shown in part (a) of FIG. 12, image-dataitems shown in parts (b) and (c) of FIG. 12 are stored in storagemediums such as memories shown in parts (d) and (e) of FIG. 12.

Finally, the entire original image is irradiated with light emitted fromthe light source, and each of line sensors reads data items of theoriginal image at its position. Then, the read image-data items areorderly stored in the memories so that a plurality of image-data itemscan be acquired, where the image-data items are shifted from each otherby as much as a single pixel in the sub-scanning direction shown inparts (a) and (b) of FIG. 13.

The image-data items with the shift in the sub-scanning direction becomeimage data of the same number of frames as that of the line sensors,where each of the line sensors includes a group of area sensors. Whenthe pixel sensors are two-dimensionally arranged and used as the areasensor for reading image data in the above-described manner, consecutiveframe-image-data items with phases shifted in the sub-scanningdirection, the consecutive frame-image-data items corresponding to aplurality of frames, can be acquired through a single read operation.

Next, a method of using the area sensor in the apparatus used in theabove-described embodiment will be described. First, the area sensorshown in FIG. 7 is mounted on the reader in a slanting position.

Each of FIGS. 14A and 14B shows exemplary mounting of the area sensorused in the above-described embodiment. FIG. 14A shows an area sensordevice 1401 and pixel sensors 1402. In the following description, thepixel sensor 1402 includes the pixel sensors of 20 pixels arranged inthe long-side direction and those of 10 pixels arranged in theshort-side direction.

Then, the area sensor is mounted, so as to be slanted in relation to thereference installation position. The position of each of the pixelsensors included in the area sensor is expressed by determining theupper left end of the area sensor to be the origin point, the long-sidedirection to be the x direction, and the short-side direction to be they direction. Namely, the coordinates of the upper left end are expressedby the equation (x, y)=(0, 0), and those of the upper right end areexpressed by the equation (x, y)=(19, 0).

Similarly, the coordinates of the lower left end are expressed by theequation (x, y)=(0, 9) and those of the lower right end are expressed bythe equation (x, y)=(19, 9).

A black frame 1403 indicates the group of the pixel sensorscorresponding to a single line, the pixel sensors constituting the areasensor device 1401. More specifically, the black frame 1403 indicates 20pixel sensors arranged in the long-side direction.

That is to say, the black frame 1403 indicates pixel sensors, where thepositions of the pixel sensors correspond to coordinates (0, 4), (1, 4),(2, 4), . . . (19, 4).

In the following description, the pixel sensors surrounded by the blackframe 1403 are referred to as a read-line sensor 1403. Similarly, ablack frame 1404 shows pixel sensors, where the positions of the pixelsensors correspond to coordinates (0, 5), (1, 5), (2, 5), . . . (19, 5).In the following description, the pixel sensors surrounded by the blackframe 1404 are referred to as a read-line sensor 1404.

In the above-described embodiment, the read unit 205 including the areasensor 213 mounted on the reader is driven in the direction of an arrowshown in FIG. 2 so that an original image placed on the original plate204 is read.

Namely, the read line sensors 1403 and 1404, where each of the sensors1403 and 1404 is a group of the pixel sensors, are handled as the linesensor. Consequently, the read operation is performed.

Next, how image data read by the read line sensors 1403 and 1404 arehandled will be described. FIG. 8 shows the original image which isread, as in the following description. That is to say, theabove-described original image corresponds to the original image 203shown in FIG. 2.

What is indicated by the grid shown in FIG. 8 corresponds to theresolution of the pixel sensors included in the read line sensors 1403and 1404. The original image is read as described above with referenceto FIGS. 9 to 13. However, since the original image is inclined at a θ°angle, image data inclined θ° is acquired.

If the area sensor is not inclined, an image shown at a positionindicated by a diagonally-shaded part shown in part (a) of FIG. 15 isread, for example. However, since the area sensor is inclined, the linesensors 1403 and 1404 detect image-data items shown in parts (b) and (c)of FIG. 15.

Then, the above-described image-data items are stored, where theinclinations thereof are left unchanged, in storage mediums such asmemories shown in parts (d) and (e) of FIG. 15. Similarly, at the sametime as when the sensor unit 205 and the light source are moved, animage shown at a position indicated by a diagonally-shaded part shown inpart (a) of FIG. 16 is read. At that time, the line sensors 1403 and1404 detect image-data items shown in parts (b) and (c) of FIG. 16.

Then, the above-described image-data items are stored in storage mediumssuch as memories shown in parts (d) and (e) of FIG. 16.

Further, the light source is moved as the read unit is moved in thesub-scanning direction so that an image shown at a position indicated bya diagonally shaded part shown in part (a) of FIG. 17 is read. At thattime, the line sensors 1403 and 1404 obtain image-data items shown inparts (b) and (c) of FIG. 17.

Then, the above-described image-data items are stored in storage mediumssuch as memories shown in parts (d) and (e) of FIG. 17.

Ultimately, the image-data items detected and read by the line sensors1403 and 1404 are data items shown in parts (a) and (b) of FIG. 18. Eachof the data items is read as image data inclined at a θ° angle. At thattime, the direction indicated by an arrow (A) and that indicated by anarrow (B) that are shown in FIG. 18 are respectively referred to as themain-scanning direction and the sub-scanning direction. On the otherhand, the direction indicated by an arrow (C) is referred to as thelandscape orientation of read-image data. Further, the directionindicated by an arrow (D) is referred to as the portrait orientation ofread-image data.

As shown in FIG. 14A, the read-line sensor 1403 and the read-line sensor1404 are physically shifted from each other in the short-side directionby as much as a single pixel sensor. Therefore, the phases of the pixelsensors included in the read-line sensor 1403 are shifted from those ofthe pixel sensors included in the read-line sensor 1404 in the long-sidedirection.

For example, a pixel sensor located at the position corresponding tocoordinates (x, y)=(15, 4), the pixel sensor being included in theread-line sensor 1403, is shifted from a pixel sensor located at theposition corresponding to coordinates (x, y)=(15, 5), the pixel sensorbeing included in the read-line sensor 1404, in the y-axis directionwhich is the short-side direction by as much as y=1. The above-describedshift causes the pixel sensors of the read-line sensors 1403 and 1404 tobe shifted from each other by as much as Δβ in the sub-scanningdirection.

On the other hand, the position of the pixel sensor of the read-linesensor 1403 in the x-axis direction which is the long-side direction isx=15, which is the same as that of the pixel sensor of the read-linesensor 1404. However, due to the inclination angle θ, the phases of theabove-described pixel sensors are shifted from each other in thehorizontal direction defined at the reference installation position byas much as Δα which is a small amount that falls within a sub pixel.Namely, even though the position of the pixel sensor of the read-linesensor 1403 is provided at the same position as that of the pixel sensorof the read-line sensor 1404 in the x-axis direction determined to bethe long-side direction, a phase shift measured in small units occurswhen the area sensor is inclined. The above-described phase shiftdepends on the inclination angle.

Therefore, the phase shifts of individual image-data items read byread-line sensors defined in the area sensor 213 vary from one linesensor to another.

More specifically, the phase of the image data shown in part (a) of FIG.18 is shifted from that of the image data shown in part (b) of FIG. 18not only in the sub-scanning direction by as much as Δβ, but also in themain-scanning direction by as much as Δα.

Although the number of the read line sensors (the read-line sensors 1403and 1404) is two in the above-described embodiment, a different numberof the read line sensors may be used without being limited to theabove-described embodiment.

The number of the pixel sensors included in the area sensor 213 may beincreased in the x-axis direction so that a large number of theread-line sensors are provided. Namely, it becomes possible to provideas many read-line sensors as the pixels that are included in the areasensor 213 and that are arranged in the x-axis direction.

The number of the read-line sensors is equivalent to that of image-dataitems obtained through a single read operation. Namely, if the read-linesensors corresponding to 30 lines are provided in the area sensor 213,it becomes possible to acquire 30 read images through a single readoperation, where each of the read images has a unique phase shift.

Thus, the area sensor is inclined so that a plurality offrame-image-data items is read at a time. According to theframe-image-data items, the positions of the original images read by thepixel sensors adjacent to one another in the scanning-long-sidedirection are shifted from one another by as much as less than a singlepixel.

Further, the area sensor 213 may be installed as shown in FIG. 14B. Thelong-side direction is the same as the horizontal direction determinedat the reference installation position. However, the area sensor 214 isinclined in the short-side direction in relation to thereference-installation position.

In that case, it becomes also possible to obtain frame-image-data itemsthrough the original image scanning at a time, where the positions ofthe original images read by the pixel sensors adjacent to one another inthe short-side direction are shifted from one another in themain-scanning direction and/or the sub-scanning direction by as much asless than a single pixel, as is the case with FIG. 14A.

That is to say, any area sensor including a plurality of sensors may beused, so long as the scanning position is moved in relatively parallelwith the original image so that frame-image data is obtained, where thepositions of original images read by sensors adjacent to one another inthe short-side direction are shifted from one another in themain-scanning direction and/or the sub-scanning direction by as much asless than a single pixel.

Further, each of the inclination angle θ shown in FIG. 14A and theinclination angle θ′ shown in FIG. 14B may be any angle so long asframe-image-data items can be obtained through a single original imagescanning, where the positions of original images read by sensorsadjacent to one another in the short-side direction are shifted from oneanother in the main-scanning direction and/or the sub-scanning directionby as much as less than a single pixel.

Further, it becomes possible to increase the number of frame-image-dataitems acquired by the sensors in the short-side direction by increasingthe number of times the original image is read in the sub-scanningdirection and the number of times sampling is performed per unit time.

Detailed Description of Printer-Image-Processing Unit 315

FIG. 6 shows the internal configuration of a printer-image-processingunit 315. A background-eliminating-processing unit 601 eliminates(removes) the background color of image data based on histogram datagenerated through the scanner-image-processing unit 312.

A monochrome-generation unit 602 converts color data into monochromedata. A log-conversion unit 603 performs luminance-to-densityconversion. The above-described log-conversion unit 603 converts RGBinput image data to CMY image data, for example.

An output-color-correction unit 604 performs output-color correction.For example, CMY input image data is converted into CMYK image data byusing a table and/or a matrix.

An output-side gamma correction unit 605 performs correction so that asignal-value data transmitted thereto becomes proportional toreflection-density-value data obtained after copying and data outputtingare performed.

A halftone correction unit 606 performs halftone processing inaccordance with the gradation number of the printer unit 12 whichoutputs data. For example, the halftone correction unit 606 convertshigh-gradation image data transmitted thereto into 2-level image dataand/or 32-level image data.

Further, each of the processing units provided in thescanner-image-processing unit 312 and/or the printer-image-processingunit 315 can output image data transmitted thereto without performingprocessing for the image data. Letting data pass through a certainprocessing unit without performing processing for the data in theabove-described manner will be expressed as “letting data pass through aprocessing unit”.

Super-Resolution-Processing Setting

Hereinafter, super-resolution-processing setting made in theabove-described embodiment will be described in detail. Here, the areasensor shown in FIG. 14A is used in the above-described embodiment.Further, the following image-processing apparatus is used. Namely, whenan original image is read through the area sensor, the image-processingapparatus can acquire frame-image data with a low resolution of about100 dpi by as much as 50 frames through a single scanning operation.Further, the above-described image-processing apparatus can generate animage with a high resolution of 200 dpi through the high-resolutionprocessing by using the low-resolution frame image data corresponding to4 frames.

Similarly, the image-processing apparatus can generate an image with ahigh resolution of 300 dpi by using low-resolution frame image data byas much as 10 frames. Further, the image-processing apparatus cangenerate an image with a high resolution of 600 dpi by usinglow-resolution frame image data by as much as 40 frames. Still further,the image-processing apparatus can generate an image with a highresolution of 1200 dpi by using low-resolution frame image data by asmuch as 100 frames. Still further, the image-processing apparatus cangenerate an image with a high resolution of 2400 dpi by usinglow-resolution frame image data by as much as 400 frames.

Here, the number of frames of low-resolution frame image data necessaryto acquire a desired resolution is not limited to the above-describedframe numbers. For example, the image-processing apparatus may generatean image with a high resolution of 1000 dpi by using low-resolutionframe image data by as much as 50 frames. Further, the image-processingapparatus may generate an image with a high resolution of 500 dpi byusing low-resolution frame image data by as much as 50 frames. Theabove-described frame number depends on the capacity of theimage-processing apparatus.

Further, not all of the acquired low-resolution frame image data can beused. For example, when the positions of pixels reading the frame imagedata acquired by adjacent sensors are shifted from one another by asmuch as a single pixel or more, namely, when the value of the phaseshift is a single pixel or more, it becomes difficult to use the frameimage data for the super resolution processing. The above-describedlow-resolution frame image data is not counted as the acquired framenumber.

FIG. 19 illustrates operations performed to executesuper-resolution-processing-mode-setting processing. A control programimplementing processing procedures shown in FIG. 19 is stored in the ROM303 and executed by the CPU 301, as described above.

First, at step S1901, an instruction to acquire SICNT, which is themaximum number of the number of low-resolution frame-image-data itemsthat can be acquired through a single scanning operation, is transmittedfrom a user via a user interface (UI).

Then, upon receiving the instruction, the UI transmits theabove-described instruction to the CPU 301 via the operation-unit I/F305, and the CPU 301 acquires the SICNT data and stores data of thenumber of frames used for the maximum frame-image data. Theabove-described maximum frame number is determined based on the numberof columns arranged in the short side direction of a sensor that will bedescribed later.

Further, the number of frame-image-data items acquired in the directionof the short side of the sensor can be increased by increasing thefrequency of reading the original image in the sub-scanning directionand the frequency of sampling data per unit time. Since frame image datawith a low resolution of about 100 dip can be acquired by as much as 50frames in the above-described embodiment, the value of the SICNT data isdetermined to be 50.

Next, at step S1902, data of an output resolution determined based onthe SICNT data acquired at step S1901 is acquired.

FIG. 20 is a schematic diagram showing an exemplary operation unitprovided to determine the output resolution. The output resolution canbe determined by the user on an output-resolution determining menu 2001.

Next, at step S1903, data of DICNT, which indicates the number of framesused in low-resolution frame image data necessary for the outputresolution of which data is acquired at step S1902, is calculated.Namely, data of the number of frames of a necessary frame image isacquired.

In the above-described embodiment, data of a table providing informationabout the correspondences between output resolutions and the number offrames used in low-resolution frame-image-data items is stored in theROM 303.

FIG. 21 shows a table provided to show information about thecorrespondence between output resolutions and the number of frames usedfor low-resolution frame-image-data items.

Next, at step S1904, the scanning operation frequency N is calculatedaccording to the following expression:

N=DICNT/SICNT.

For example, when the output resolution is 2400 dpi, the expressionDICNT=400 holds. Therefore, the scanning operation frequency N iscalculated, as shown in the following expression:

N=400/50=8.

Next, at step S1905, the value of a scan counter CNT is reset to 0, andthe value of the scan counter CNT and the scanning operation frequency Nare compared to each other at step S1906.

If it is determined that the value of the scan counter CNT is less thanthat of the scanning operation frequency N (YES at step S1906), data ofan original is read at step S1907, the value of the scan counter CNT isincremented by 1 at step S1908, and processing then returns to stepS1906. Thus, the read frequency is controlled based on a result of theread-frequency calculation.

On the other hand, if it is determined that the value of the scancounter CNT is not less than that of the scanning operation frequency N(NO at step S1906), it is determined that the original-data reading isfinished. Then, the inclination of acquired frame image data iscorrected at step S1909 and the super-resolution processing is executedat step S1910 so that the processing procedures are finished.

The inclination of the acquired frame image data is corrected, asdescribed below. The inclination of the acquired frame image data isequivalent to the inclination angle θ of the area sensor, as describedabove.

The above-described inclination angle θ of the area sensor 213 is avalue that can be acquired at the time when the area sensor 213 ismounted on the read unit 205 during the process of assembling an MFPincluding the area sensor 213.

Data of the above-described inclination angle θ is stored in a storagearea provided in the MFP as data on a value unique to the mounteddevice.

The affine transform is performed by using the above-described angleinformation so that acquired inclined frame-image data is rotated. Atthat time, the frame image data is rotated by as much as an inclinationfrom the reference installation position so that the inclination of theframe image data is corrected. If the coordinates of the frame imagedata that had not been subjected to the affine transform and those ofthe frame image data that had been subjected to the affine transform aredetermined to be (X, Y) and (X′, Y′), respectively, and the rotationangle (the inclination angle of the area sensor in the above-describedembodiment) is determined to be θ, frame image data of which inclinationis corrected through affine-transform processing is obtained, as shownby Equation 1.

$\begin{matrix}\left\lbrack {{Equation}\mspace{14mu} 1} \right\rbrack & \; \\{\left\lbrack {X^{\prime},Y^{\prime}} \right\rbrack = {\left\lbrack {X,Y} \right\rbrack \begin{bmatrix}{\cos \; \theta} & {\sin \; \theta} \\{{- \sin}\; \theta} & {\cos \; \theta}\end{bmatrix}}} & \left( {{Equation}\mspace{14mu} 1} \right)\end{matrix}$

-   X′, Y′: coordinate position attained after transform-   X, Y: coordinate position attained before transform

The frame image data obtained through the affine transform becomeslow-resolution frame image data with a corrected inclination.

Here, the method of correcting the inclination is not limited to theaffine transform. Namely, any method can be used so long as theinclination of the frame image data can be corrected therethrough. If asensor which can obtain the frame image data in which the positions atwhich the sensors adjacent to one another in the short-side directionread data are shifted from one another in the main scanning directionand/or the sub scanning direction by as much as less than a singlepixel, as shown in FIG. 14B, can obtain frame image data with noinclination, the above-described processing is unnecessary.

Then, the super-resolution processing is executed at step S1910 by usingframe image data with corrected inclination, the frame image datacorresponding to a plurality of frames, so that the processing isfinished. Here, the steps S1909 and S1910 are not in particular order.The super-resolution processing may be executed first so that theinclination of high-resolution image data acquired through thesuper-resolution processing is corrected after the super-resolutionprocessing.

The super-resolution processing is executed to generate high-resolutionimage data by using a plurality of frame-image-data items with phasesshifted from one another by as much as less than a single pixel in themain scanning direction and the sub scanning direction for each RGBchannel, as shown in FIG. 5.

For example, frame-image-data items 501, 502, 503, and 504 that areshown in FIG. 5 are shifted from one another by as much as one-half of apixel by using four image-data items. Therefore, it becomes possible toobtain a high-resolution image having a pixel density which is fourtimes higher than that of original image data 505 by using theabove-described image data.

The high-resolution processing executed at that time will be morespecifically described with reference to FIGS. 22 and 23. FIG. 22 showslow-resolution frame-image data used for the high-resolution processingand image data that had been subjected to the high-resolutionprocessing. FIG. 22 shows an original, and reference low-resolutionimage data F0 and object low-resolution image-data items F1 to F3 thatare acquired by reading the original through the area sensor. Each ofdotted rectangles surrounding the original indicates an area in whichthe reference low-resolution image data F0 is read by the area sensor,and solid-line rectangles indicate areas in which the objectlow-resolution image-data items F1 to F3 are respectively read by thearea sensor.

In the above-described embodiment, the amount of shift in themain-scanning direction is expressed as “um”, and that of shift in thesub-scanning direction is expressed as “vm”. Further, the shift amountsof object low-resolution image-data items Fn (n=1 to 3) are expressed as“umn”, and “vmn”. For example, as shown in FIG. 22, the objectlow-resolution image-data item F1 is shifted from the referencelow-resolution image data F0 in the sub-scanning direction, and theshift amount is expressed as um1, vm1. Similarly, the shift amounts ofthe object low-resolution image-data items F2 and F3 are respectivelyexpressed as um2, vm2, and um3, vm3.

The correction amounts un, vn of the object low-resolution image-dataitems Fn are calculated based on the image data of the referencelow-resolution image data F0 and the image data of the objectlow-resolution image-data items F1 to F3. The above-describedcalculation is performed according to a predetermined calculation methodusing information about the inclination of the area sensor, where theinformation is stored in the ROM 303 in advance.

FIG. 22 schematically shows that the shift of each of the objectlow-resolution image-data items occurs in pixels. However, when data isread by the area sensor of the above-described embodiment, the phaseshift occurs in the main scanning direction and the sub scanningdirection by as much as less than a single pixel. It becomes possible tomake a high-resolution image by using the above-described small shift.

Therefore, of the pixels included in a generatedsuper-resolution-processing image, there are pixels that are notincluded in the reference low-resolution image data and the objectlow-resolution image data.

For the above-described pixels, a high-resolution image is generatedwhile performing a merge by performing predetermined interpolationprocessing by using pixel data indicating the values of pixels providedaround the generation pixel. As the interpolation processing, thebilinear interpolation method, the bicubic method, the nearest neighbormethod, and so forth can be used.

For example, FIG. 23 shows the case where the interpolation processingis performed according to the bilinear interpolation method. First, dataon the nearest pixel 1802 nearest the position of a generation pixel1801 is extracted from the reference low-resolution image data and theobject low-resolution image data. Then, four pixels surrounding thegeneration-pixel position are determined to be surrounding pixels 1802,1803, 1804, and 1805. Further, values obtained by assigningpredetermined weights to the data values of the surrounding pixels areaveraged, and the data value of the generation pixel is obtained, asshown in the following equation.

f(x, y)=[|x1−x|{|y1−y|f(x0, y0)+|−y−y0|f(x0, y1)}+|x−x0|{|y1−y|f(x,y0)+|y−y0|f(x1, y1)}]/|x1−x0||y1−y0|

The above-described processing procedures are performed for each of thegeneration-pixel positions so that a super resolution image with aresolution two times higher than that of an original image is obtained,as shown in FIG. 20. Here, the resolution may not be two times higherthan that of the original image. Namely, various magnifications can beused. Further, as the values of low-resolution image-data items are usedfor the interpolation processing, the definition of a super-resolutionimage obtained through the interpolation processing is increased.

Original Read Processing

Original-read processing described at step S1905 shown in FIG. 19 willbe specifically described. In the above-described embodiment, theoriginal is read through the area sensor.

FIG. 24 shows an exemplary area sensor provided in the scanner unit 11used in the above-described embodiment. In that case, the area sensor isdivided into 50 parts in the sub-scanning direction, as indicated by asection 2401. Further, frame image data with a resolution of 100 dpi canbe acquired in bands by as much as 50 frames at the maximum.

Control is performed so that data is read by determining one of theabove-described bands to be a single line sensor. Further, in how manyparts the above-described area should be divided, that is, how many linesensors are to be prepared are arbitrarily determined. In theabove-described embodiment, the installed area sensor is divided into 50parts and used, as 50 line sensors.

Therefore, if the value of the scanning operation frequency calculatedat step S1904 shown in FIG. 19 is at least two, band-data items 0 to 49that are acquired through the first scanning operation are respectivelystored, as frame image-data items ID0 to ID49.

Then, band-data items 50 to 99 that are acquired through the secondscanning operation are respectively stored, as frame image-data itemsID50 to ID99. Thus, the frame image data is managed for each scanningoperation based on the frame image data ID, which makes it possible toacquire low-resolution frame image data used to reproduce an ultimatelyspecified output resolution.

Further, in the above-described embodiment, the scanning operation iscontinuously executed for a single page of the original image. At thattime, the original image is fixed on the original plate and the lightsource is moved so that the original image is read. The above-describedconfiguration is determined to be the first area-sensor system.

When the scanning operation is executed at least two times by using theabove-described first area-sensor system, the original image placed onthe original plate may not be deliberately moved. Although the positionwhere the original image is placed is not changed, signals acquiredthrough the first and second scanning operations are different from eachother due to the optical characteristic of the light source and theprecision of control performed by the device while the scanner executesscanning.

If the signal difference falls within a sub pixel, the datacorresponding thereto may be adopted as low-resolution frame image dataand used for the subsequent processing. Therefore, while theabove-described scanning operations are performed, a warning messageinstructing the user not to move the original may be displayed on theUI. Otherwise, data on the number of remaining scanning operations maybe displayed on the UI.

Aside from that, original images are often placed in an automaticdocument feeder (hereinafter referred to as an “ADF”), so as to scan asingle set of documents including a plurality of pages.

At that time, the light source is fixed and the original images aremoved so that the original images are read. The above-describedconfiguration is determined to be the second area-sensor system.

In that case, after a single scanning operation is executed for thesingle set of documents, the user may be encouraged to place another setof documents in the ADF. In that case, data on the number of remainingscanning operations may be displayed on the UI.

In the case where the first read operation is executed on the originalplate through the first area sensor system and the second read operationis executed by using the ADF through the second area sensor system inthe above-described manner, the value of signals acquired at the readtime through the first and second read operations are different fromeach other due to the precision of controlling the positions where thedevice reads data and the optical characteristic of the light source.

If the signal difference falls within a sub pixel, the datacorresponding thereto may be adopted as low-resolution frame image data,and used for the subsequent processing.

Further, if the user stops performing the scanning operation inprogress, high-resolution image data can be generated by using aplurality of low-resolution frame-image-data items obtained through atleast one scanning operation performed by then.

The scanning operation number is determined in accordance with the setoutput resolution through the above-described processing procedures.Consequently, it becomes possible to output high-resolution image datathat can be reproduced with difficulty through a single scanningoperation.

Further, the above-described term “output” indicates converting ascanned image into a high-resolution image and actually printing theimage data on a sheet. In addition, the above-described term “output”indicates storing the image data subjected to the high-resolutiontransform in the image-processing apparatus without printing the imagedata on a sheet.

According to the first embodiment, the scanning operation frequency isdetermined based on the set output resolution and the scanning operationis performed with the determined frequency so that high-resolution imagedata is output.

A second embodiment of the present invention illustrates the case wherean image-processing apparatus on which the ADF is mounted performs thescanning operation a plurality of times. In that case, the firstscanning operation is performed as scanning performed while a movingoriginal fed from the ADF is read. Further, the same processingprocedures as those performed in the first embodiment are designated bythe same reference numerals and redundant schematic descriptions thereofare omitted.

FIG. 25 shows an ADF 2501, which is a part of the scanner unit 11described in FIG. 1, and FIG. 26 shows the configuration of a scannermain body 2601. The scanner unit 11 is provided withpressure-plate-reading mode in which an original image is placed stillon the glass of the original plate and an optical system is moved sothat the original image is read and moving-original-reading-mode inwhich the optical system is stopped and the original image is moved sothat the original image is read.

FIG. 27 schematically illustrates an example of operations that can beperformed to execute the moving-reading scanning and thepressure-plate-reading scanning during the original-image readingperformed at step S1907 shown in FIG. 19 according to the secondembodiment. A control program achieving processing procedures shown inFIG. 27 is stored in the ROM 303 and executed by the CPU 301, asdescribed above.

First, at step S2701, the scanning operation frequency N calculated atstep S1904 is acquired.

Next, it is determined whether the value of the scanning operationfrequency N acquired at step S2701 is at least two at step S2702.

If it is determined that the value of the scanning operation frequency Nis at least two (YES at step S2702), processing continues at step S2703,otherwise (NO at step S2702) processing continues at step S2705). AtStep S2703, the first scanning operation is set to themoving-original-reading mode. Next, the ADF 2501 conveys an originalimage placed therein to the glass of the original plate of the scannermain body 2601 through a conveying roller.

In that case, an original image optically scanned in themoving-original-reading mode through the scanner main body 2601 is readas a plurality of low-resolution image-data items at step S2704.

After that, the original image is conveyed onto the glass of theoriginal plate of the scanner main body 2601. Next, the remainingscanning operations are determined to be executed in thepressure-plate-reading mode at step S2705.

Then, the first mirror unit 2602 and the second mirror unit 2603 thatare provided in the scanner main body are temporarily returned to thehome position where a home position sensor 2604 is provided.

Then, an original illumination lamp 2605 is turned on and the originalimage is irradiated with the light thereof. The light reflected from theoriginal image passes through a lens 2609 via the first mirror 2606provided in the first mirror unit 2602 and the second and third mirrors2607 and 2608 that are provided in the second mirror unit so that animage is formed on the area sensor 2610. After that, data on the imageis transmitted to the area sensor 2610, as an optical signal, and theoriginal-image reading is executed at step S2706 so that the processingprocedures are terminated.

Thus, after the first original image reading is performed through theADF, the original image is automatically conveyed to a position definedfor the pressure-plate-reading mode even though the user does not placethe original image at a different position, which saves the user timeand trouble.

On the other hand, if it is determined that the value of the scanningoperation frequency N is one at step S2702, the original reading isexecuted in the pressure-plate-reading mode while the processingprocedures corresponding to steps S2703 and S2704 are not executed.

Thus, when the scanning operation is performed a plurality of timesbased on the set output resolution, the first scanning operation isperformed as scanning performed while reading a moving original fed fromthe ADF.

After that, the pressure-plate-reading scanning is executed, which makesit possible to output high-resolution image data that can be reproducedwith difficulty through a single scanning operation. Particularly, ifthe scanning operation is performed twice in the second embodiment,reading processing can be performed with a speed higher than that of asystem that performs the pressure-plate-reading scanning twice.

Further, the present invention can be used for a system including aplurality of units (e.g., a computer, an interface device, a reader, aprinter, and so forth) and/or an apparatus including a single unit (animage-processing apparatus, a printer, a fax machine, and so forth).

Further, the present invention can also be implemented by a computer (ora central processing unit (CPU) and/or a microprocessing unit (MPU))reading and executing program code implementing the procedural steps ofthe flowcharts described in the above-described embodiments from astorage medium storing the program code. In that case, the program coderead from the storage medium implements the functions of theabove-described embodiments. The program code and the storage mediumstoring the program code constitute an embodiment of the presentinvention.

The storage medium provided to supply the program code may be, forexample, a floppy (registered trademark) disk, a hard disk, an opticaldisk, a magneto-optical disk, a compact disk (CD)-read only memory(ROM), a CD-recordable (R), a magnetic tape, a nonvolatile memory card,a ROM, etc.

Further, not only by the computer reading and executing the programcode, but also by the computer executing part of or the entire processutilizing an operating system (OS), etc. running on the computer basedon instructions of the program code, the functions of theabove-described embodiments may be achieved. The latter is also one ofembodiments of the present invention.

In another embodiment of the present invention, the program code readfrom the storage medium may be written into a memory of a functionexpansion board inserted in the computer and/or a function expansionunit connected to the computer. At that time, the functions of theabove-described embodiments are realized by executing part of or theentire process by a CPU, etc. of the function expansion board and/or thefunction expansion unit based on instructions of the program code.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all modifications and equivalent structures and functions.

This application claims the benefit of Japanese Patent Application No.2007-330978 filed Dec. 21, 2007 and Japanese Patent Application No.2008-318563 filed Dec. 15, 2008, which are hereby incorporated byreference herein in their entirety.

1. An image-processing apparatus comprising: an area sensor unitconfigured to read image data items corresponding to a plurality offrames from an original image, the image data items having at least oneshift corresponding to less than a single pixel; a correction unitconfigured to correct an inclination of each of the image data itemsacquired through the area-sensor unit; a high-resolution conversion unitconfigured to acquire image data with a resolution higher than aresolution used during data reading by performing interpolationprocessing by using the corrected image data items; a maximum framenumber storage unit configured to store data of a maximum number of atleast one frame of the image data items acquired through the area sensorunit; a resolution setting unit configured to set a resolution used tooutput the original image; a necessary frame number acquisition unitconfigured to acquire a number of at least one frame necessary toperform the high resolution conversion based on a result of the settingmade by the resolution determining unit; a read frequency calculationunit configured to calculate a frequency of reading the original imagethrough the necessary frame number acquisition unit and the maximumframe number storage unit; and a read frequency control unit configuredto read the frequency determined by the read frequency calculation unitand the original image.
 2. The image processing apparatus according toclaim 1, wherein, when the read frequency calculation unit determinesthat it is necessary to read the original image a plurality of times, afrequency of remaining reading determined to be necessary is displayedon a user interface.
 3. The image processing apparatus according toclaim 1, wherein, when reading which is to be performed with thecalculated frequency is stopped, the high resolution conversion isexecuted by using image data read before the reading is stopped.
 4. Theimage processing apparatus according to claim 1, wherein the area sensorunit includes: a first read unit in which the original image is fixedonto an original plate and a light source is moved so that the originalimage is read; and a second read unit in which the light source is fixedand the original image is moved so that the original image is read, anda read control unit configured to perform control so that when a valueof the calculated reading frequency is two ore more, the original imageis read by using the first and second read units, and when the value ofthe calculated reading frequency is not two or more, the original imageis read by using either the first read unit or the second read unit. 5.The image processing apparatus according to claim 2, wherein the areasensor unit includes: a first read unit in which the original image isfixed onto an original plate and a light source is moved so that theoriginal image is read; and a second read unit in which the light sourceis fixed and the original image is moved so that the original image isread, and a read control unit configured to perform control so that whena value of the calculated reading frequency is two ore more, theoriginal image is read by using the first and second read units, andwhen the value of the calculated reading frequency is not two or more,the original image is read by using either the first read unit or thesecond read unit.
 6. The image processing apparatus according to claim3, wherein the area sensor unit includes: a first read unit in which theoriginal image is fixed onto an original plate and a light source ismoved so that the original image is read; and a second read unit inwhich the light source is fixed and the original image is moved so thatthe original image is read, and a read control unit configured toperform control so that when a value of the calculated reading frequencyis two ore more, the original image is read by using the first andsecond read units, and when the value of the calculated readingfrequency is not two or more, the original image is read by using eitherthe first read unit or the second read unit.
 7. The image-processingapparatus according to claim 1, wherein a read area of the area sensoris divided into an arbitrary number of areas and the original image isread for each of the areas.
 8. The image-processing apparatus accordingto claim 2, wherein a read area of the area sensor is divided into anarbitrary number of areas and the original image is read for each of theareas.
 9. The image-processing apparatus according to claim 3, wherein aread area of the area sensor is divided into an arbitrary number ofareas and the original image is read for each of the areas.
 10. Theimage-processing apparatus according to claim 4, wherein a read area ofthe area sensor is divided into an arbitrary number of areas and theoriginal image is read for each of the areas.
 11. The image-processingapparatus according to claim 5, wherein a read area of the area sensoris divided into an arbitrary number of areas and the original image isread for each of the areas.
 12. The image-processing apparatus accordingto claim 6, wherein a read area of the area sensor is divided into anarbitrary number of areas and the original image is read for each of theareas.
 13. An image processing method used in an image processingapparatus including: an area sensor unit configured to read image dataitems corresponding to a plurality of frames from an original image, theimage data items having at least one shift corresponding to less than asingle pixel; a correction unit configured to correct an inclination ofeach of the image data items acquired through the area-sensor unit; anda high resolution conversion unit configured to acquire image data witha resolution higher than a resolution used during data reading byperforming interpolation processing by using the corrected image dataitems, the image-processing method comprising: a maximum frame numberstorage step provided to store data of a maximum number of at least oneframe of the image data items acquired through the area sensor unit; aresolution setting step provided to set a resolution used to output theoriginal image; a necessary frame number acquisition step provided toacquire a number of at least one frame necessary to perform the highresolution conversion based on a result of the setting made at theresolution determining step; a read frequency calculation step providedto calculate a frequency of reading the original image through thenecessary frame number acquisition step and the maximum frame numberstorage step; and a read frequency control step provided to read thefrequency determined at the read frequency calculation step and theoriginal image.
 14. The image processing method according to claim 13,wherein, when it is determined that reading the original image aplurality of times is necessary at the read frequency calculation step,a frequency of remaining reading determined to be necessary is displayedon a user interface.
 15. The image processing method according to claim13, wherein, when reading which is to be performed with the calculatedfrequency is stopped, the high resolution conversion is executed byusing image data read before the reading is stopped.
 16. The imageprocessing method according to claim 13, wherein the area sensor unitincludes: a first read unit in which the original image is fixed onto anoriginal plate and a light source is moved so that the original image isread; and a second read unit in which the light source is fixed and theoriginal image is moved so that the original image is read, and a readcontrol unit configured to perform control so that when a value of thecalculated reading frequency is two ore more, the original image is readby using the first and second read units, and when the value of thecalculated reading frequency is not two or more, the original image isread by using either the first read unit or the second read unit. 17.The image processing apparatus according to claim 14, wherein the areasensor unit includes: a first read unit in which the original image isfixed onto an original plate and a light source is moved so that theoriginal image is read; and a second read unit in which the light sourceis fixed and the original image is moved so that the original image isread, and a read control unit configured to perform control so that whena value of the calculated reading frequency is two ore more, theoriginal image is read by using the first and second read units, andwhen the value of the calculated reading frequency is not two or more,the original image is read by using either the first read unit or thesecond read unit.
 18. The image processing apparatus according to claim15, wherein the area sensor unit includes: a first read unit in whichthe original image is fixed onto an original plate and a light source ismoved so that the original image is read; and a second read unit inwhich the light source is fixed and the original image is moved so thatthe original image is read, and a read control unit configured toperform control so that when a value of the calculated reading frequencyis two ore more, the original image is read by using the first andsecond read units, and when the value of the calculated readingfrequency is not two or more, the original image is read by using eitherthe first read unit or the second read unit.
 19. The image-processingmethod according to claim 13, wherein a read area of the area sensor isdivided into an arbitrary number of areas and the original image is readfor each of the areas.
 20. The image-processing method according toclaim 14, wherein a read area of the area sensor is divided into anarbitrary number of areas and the original image is read for each of theareas.
 21. The image-processing method according to claim 15, wherein aread area of the area sensor is divided into an arbitrary number ofareas and the original image is read for each of the areas.
 22. Theimage-processing method according to claim 16, wherein a read area ofthe area sensor is divided into an arbitrary number of areas and theoriginal image is read for each of the areas.
 23. The image-processingmethod according to claim 17, wherein a read area of the area sensor isdivided into an arbitrary number of areas and the original image is readfor each of the areas.
 24. The image-processing method according toclaim 18, wherein a read area of the area sensor is divided into anarbitrary number of areas and the original image is read for each of theareas.
 25. An image-processing apparatus comprising: an area sensor unitincluding a plurality of sensors including a first sensor and a secondsensor adjacent to the first sensor, where the first and second sensorsare arranged so that a first read position of the first sensor, where anoriginal image is read at the first read position, is shifted from asecond read position of the second sensor, where the original image isread at the second read position, by as much as less than a singlepixel; a high resolution conversion unit configured to acquire imagedata with a resolution higher than a resolution used during the readingtime by performing interpolation processing by using image data itemscorresponding to a plurality of frames, the image data items being readthrough the area sensor unit; a maximum frame number storage unitconfigured to store data of a maximum number of at least one frame ofthe image data items acquired through the area sensor unit; a resolutionsetting unit configured to set a resolution used to output the originalimage; a necessary frame number acquisition unit configured to acquire anumber of at least one frame of image data necessary to perform the highresolution conversion based on a result of the setting made by theresolution determining unit; a read frequency calculation unitconfigured to calculate a frequency of reading the original imagethrough the necessary frame number acquisition unit and the maximumframe number storage unit; and a read frequency control unit configuredto read the frequency determined by the read frequency calculation unitand the original image.
 26. An image processing method used in an imageprocessing apparatus including: an area sensor unit including aplurality of sensors including a first sensor and a second sensoradjacent to the first sensor, where the first and second sensors arearranged so that a first read position of the first sensor, where anoriginal image is read at the first read position, is shifted from asecond read position of the second sensor, where the original image isread at the second read position, by as much as less than a singlepixel; and a high resolution conversion unit configured to acquire imagedata with a resolution higher than a resolution used during the readingtime by performing interpolation processing by using image data itemscorresponding to a plurality of frames, the image data items being readthrough the area sensor unit, the image-processing method comprising: amaximum frame number storage step provided to store data of a maximumnumber of at least one frame of the image data items acquired throughthe area sensor unit; a resolution setting step provided to set aresolution used to output the original image; a necessary frame numberacquisition step provided to acquire a number of at least one frame ofimage data necessary to perform the high resolution conversion based ona result of the setting made at the resolution determining step; a readfrequency calculation step provided to calculate a frequency of readingthe original image through the necessary frame number acquisition stepand the maximum frame number storage step; and a read frequency controlstep provided to read the frequency determined at the read frequencycalculation step and the original image.
 27. A program making animage-processing apparatus including: an area sensor unit configured toread image data items corresponding to a plurality of frames from anoriginal image, the image data items having at least one shiftcorresponding to less than a single pixel; a correction unit configuredto correct an inclination of each of the image data items acquiredthrough the area-sensor unit; and a high resolution conversion unitconfigured to acquire image data with a resolution higher than aresolution which is used during data reading by performing interpolationprocessing by using the corrected image data items, execute: a maximumframe number storage step provided to store data of a maximum number ofat least one frame of the image data items acquired through the areasensor unit; a resolution setting step provided to set a resolution usedto output the original image; a necessary frame number acquisition stepprovided to acquire a number of at least one frame necessary to performthe high resolution conversion based on a result of the setting made atthe resolution determining step; a read frequency calculation stepprovided to calculate a frequency of reading the original image throughthe necessary frame number acquisition step and the maximum frame numberstorage step; and a read frequency control step provided to read thefrequency determined at the read frequency calculation step and theoriginal image.
 28. A computer readable storage medium storing theprogram according to claim 27.